1. Field of Invention
This invention is directed to image correlation systems.
2. Description of Related Art
Various known devices use images acquired by a sensor array, and correlation between images acquired by the sensor array, to determine deformations and/or displacements. For example, one class of such devices is based on acquiring a speckle image generated by illuminating an optically rough surface with a light source. Generally, the light source is a coherent light source, such as a laser-generating light source. Such laser-generating light sources include a laser, laser diode and the like. After the optically rough surface is illuminated by the light source, the light scattered from the optically rough surface is imaged onto an optical sensor. The optical sensor can be a charge-couple device (CCD), a semi-conductor image sensor array, such as a CMOS image sensor array, or the like.
Prior to displacing or deforming the optically rough surface, a first initial speckle image is captured and stored. Then, after displacing or deforming the optically rough surface, a second or subsequent speckle image is captured and stored. Conventionally, the first and second speckle images are then compared in their entireties on a pixel-by-pixel basis. In general, a plurality of comparisons are performed. In each comparison, the first and second speckle images are offset, or spatially translated, relative to each other. Between each comparison, the amount of offset, or spatial translation, is increased by a known amount, such as one image element, or pixel, or an integer number of image elements or pixels.
In each comparison, the image value of a particular pixel in the reference image is multiplied by, subtracted from, or otherwise mathematically used in a function with, the image value of the corresponding second image pixel, where the corresponding second image pixel is determined based on the amount of offset. The value resulting from each pixel-by-pixel operation is accumulated with values resulting from the operation performed on every other pixel of the images to determine a correlation value for that comparison between the first and second images. That correlation value is then, in effect, plotted against the offset amount, or spatial translation position, for that comparison to determine a correlation function value point. The offset having the greatest correlation between the reference and first images will generate a peak, or a trough, depending on how the pixel-by-pixel comparison is performed, in the plot of correlation function value points. The offset amount corresponding to the peak or trough represents the amount of displacement or deformation between the first and second speckle images.
U.S. patent application Ser. No. 09/584,264, which is incorporated herein by reference in its entirety, discloses a variety of different embodiments of a speckle-image-based optical transducer. As disclosed in the 264 application, such image-based correlation systems can move the surface being imaged relative to the imaging system in one or two dimensions. Furthermore, the surface being imaged does not need to be planar, but can be curved or cylindrical. Systems having two dimensions of relative motion between the surface being imaged and the imaging system can have the surface being imaged effectively planar in one dimension and effectively non-planar in a second dimension, such as, for example, a cylinder which can rotate on its axis passed the imaging systems, while the cylindrically surface is translated past the imaging system along its axis.
U.S. patent application Ser. No. 09/731,671, which is incorporated herein by reference in its entirety, discloses systems and methods for high-accuracy displacement determination in a correlation-based position transducer. In the 671 application, a system is provided that estimates the sub-pixel displacement of images in correlation-based position transducers and the like. The system then rejects the systematic displacement estimation errors present when conventional sub-pixel estimation methods are applied to a number of correlation function value points, especially when the correlation function value points are arranged asymmetrically.
However, in the above-described conventional image correlation systems, the computational loads required to determine the correlation function value over the entire image for each offset position are often extremely high. Accordingly, in “Hierarchical Distributed Template Matching” by M. Hirooka et al., Machine Vision Applications and Industrial Inspection V, Proceedings of SPIE, Feb. 10–11, 1997, San Jose, Calif., and “Coarse-Fine Template Matching” by A. Rosenfeld et al., in IEEE Transactions on Systems, Man and Cybernetics, pages 104–107, February 1977, various techniques are described that reduce the computational load by reducing the resolution of the images to be correlated. In particular, in both of these papers, the image resolution is reduced by averaging the image values of a number of pixels to create a “shrunken” image having a reduced number of pixels. The image correlation is then performed on a pixel-by-pixel basis for each offset position for the reduced resolution images. Once the general area of the greatest correlation is identified, the original, full-resolution images are compared on a pixel-by-pixel basis for each offset position in this area only.
Similarly, in “A Two-Stage Cross Correlation Approach To Template Matching” by A. Goshtasby et al., IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 6, No. 3, May 1984, a different two-stage technique is disclosed. In this paper, rather than reducing the resolution of the entire image, as in Rosenfeld et al. and Hirooka et al., a limited number of the pixels in the images to be correlated are compared for every image offset position to generate a correlation function. Like Hirooka et al. and Rosenfeld et al., a reduced number of pixels are used in the comparison. However, unlike Hirooka et al. and Rosenfeld et al., the pixels used are at full resolution but do not represent the entire image to be compared. As in Hirooka et al. and Rosenfeld et al., in this technique, once an area of high correlation is identified using the reduced number of pixels only, that area is further analyzed using all of the pixels of the images to be compared for each offset position.
In contrast to the reduced resolution technique disclosed in Hirooka et al. and Rosenfeld et al., and in contrast to the limited portion of the full resolution image technique used in Goshtasby et al., in “Advances in Picture Coding” by H. G. Musmann et al., Proceedings of the IEEE, Vol. 73, No. 4, April 1985, pages 523–548, two techniques are discussed that search a number of coarsely-spaced search points around a center search point. At each such search point, a full image correlation value is determined. Then, some analysis of the obtained correlation values is performed. These analyses generally indicate the direction of the correlation peak or trough relative to the coarsely-spaced search points. The coarsely-spaced search point that lies closest to the direction of the correlation peak or trough is then selected as the center point around which a further number of coarsely-spaced search points will be selected. This procedure proceeds iteratively until the correlation peak or trough is identified. However, at no time is any reduced representation of the images such as those disclosed in Hirooka et al., Rosenfeld et al. or Goshtasby et al. used. Likewise, while the techniques disclosed in Musmann et al. collapse the sparsely spaced search points around the central point as the central point approaches the correlation peak or trough, each iteration uses the same number of coarsely-spaced points.